against one's better judgment
Despite serious misgivings or objections, as in
Against my better judgment, I told her to come whenever she pleased
.